{ "cells": [ { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Parameter inference using neural networks\n", "\n", "Tom Charnock\n", "\n", "Institut d'Astrophysique de Paris\n", "


\n", "\n", "
\n", "
\n", " \"Sorbonne\n", "
\n", "
\n", " \"ANR\"\n", "
\n", "
\n", " \"IAP\"\n", "
\n", "
\n", " \"CNRS\"\n", "
\n", "
\n", " \"Aquila\"\n", "
\n", "
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "#### Joint distribution of parameters, $\\boldsymbol{\\theta}$, and possible data, ${\\bf d}$,
which can be generated by a model, $\\mathcal{M}:\\boldsymbol{\\theta}\\to{\\bf d}$\n", "
\n", "
\n", "\n", "
\n", "
\n", "
$$\n", "\\small\\mathcal{P}(\\boldsymbol{\\theta}|{\\bf d},\\mathcal{M}) = \\frac{\\mathcal{L}({\\bf d}|\\boldsymbol{\\theta},\\mathcal{M})p(\\boldsymbol{\\theta}|\\mathcal{M})}{p({\\bf d}|\\mathcal{M})}\n", "$$\n", "
\n", "

\n", "Obligatory Bayes Theorem\n", "

\n", "Likelihood\n", " $$\\mathcal{L}({\\bf d}|\\boldsymbol{\\theta}^*,\\mathcal{M})$$\n", "
\n", "Posterior\n", " $$\\mathcal{P}(\\boldsymbol{\\theta}|{\\bf d}^*,\\mathcal{M})$$\n", "
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Parameter inference using neural networks" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### It's common to try and predict model parameters by regression\n", "
\n", "\n", "
\n", "
\n", "$$\\mathbb{NN}(\\boldsymbol{\\omega},\\boldsymbol{\\alpha}) : {\\bf d}\\rightarrow\\boldsymbol{\\tau}$$\n", "An approximation to a model, $\\mathcal{M} : {\\bf d}\\rightarrow\\boldsymbol{\\theta}$\n", "
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## How likely are we to obtain
any particular output from the network\n", "

\n", "\n", "
\n", "$$\\Lambda(\\boldsymbol{\\theta},\\boldsymbol{\\tau})\\propto-\\ln\\mathcal{L}(\\boldsymbol{\\theta}|{\\bf d},\\boldsymbol{\\omega},\\boldsymbol{\\alpha})$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### The posterior becomes the posterior predictive density\n", "\n", "$$\\mathcal{P}(\\boldsymbol{\\theta}|{\\bf d}) = \\int d\\boldsymbol{\\omega}d\\boldsymbol{\\alpha}~\\mathcal{L}(\\boldsymbol{\\theta}|{\\bf d},\\boldsymbol{\\omega},\\boldsymbol{\\alpha})\\mathcal{P}(\\boldsymbol{\\omega},\\boldsymbol{\\alpha})$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "
\n", " $\\mathcal{P}(\\boldsymbol{\\theta}|{\\bf d})$ - Posterior predictive density
\n", " How likely are the true parameters given some data?

\n", "
\n", "

\n", "
\n", " $\\mathcal{L}(\\boldsymbol{\\theta}|{\\bf d},\\boldsymbol{\\omega},\\boldsymbol{\\alpha})$ - Likelihood
\n", " How likely are the parameters to be generated by a particular network?\n", "
\n", "
\n", " $\\mathcal{P}(\\boldsymbol{\\omega},\\boldsymbol{\\alpha})$ - Probability density
\n", " What is the probability of obtaining a particular network with particular parameter values?\n", "
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### The chance of getting any parameter value from any network given some data is almost equal\n", "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### So all the information comes from prior knowledge of the weights and hyperparameters\n", "\n", "$$\\mathcal{P}(\\boldsymbol{\\theta}|{\\bf d})= \\int d\\boldsymbol{\\omega}d\\boldsymbol{\\alpha}~\\mathcal{L}(\\boldsymbol{\\theta}|{\\bf d},\\boldsymbol{\\omega},\\boldsymbol{\\alpha})\\mathcal{P}(\\boldsymbol{\\omega},\\boldsymbol{\\alpha})$$\n", "

\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Where does the information about the weights and hyperparameters come from?" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Training and validation data" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "$$\\boldsymbol{\\{}{\\bf d},\\boldsymbol{\\theta}\\boldsymbol{\\}}^\\textrm{train}\\equiv\\{{\\bf d}^\\textrm{train}_i, \\boldsymbol{\\theta}^\\textrm{train}_i|i\\in[1,n_\\textrm{train}]\\}$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "$$\\boldsymbol{\\{}{\\bf d},\\boldsymbol{\\theta}\\boldsymbol{\\}}^\\textrm{val}\\equiv\\small\\{{\\bf d}^\\textrm{val}_i, \\boldsymbol{\\theta}^\\textrm{val}_i|i\\in[1,n_\\textrm{val}]\\}$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### The prior distribution of weights and hyperparameters
becomes the posterior\n", "$$\\begin{align*}\n", " &\\small\\mathcal{P}(\\boldsymbol{\\omega},\\boldsymbol{\\alpha}|\\boldsymbol{\\{}{\\bf d},\\boldsymbol{\\theta}\\boldsymbol{\\}}^\\textrm{train},\\boldsymbol{\\{}{\\bf d},\\boldsymbol{\\theta}\\boldsymbol{\\}}^\\textrm{val})\\propto\\\\\n", " &\\small\\phantom{hello}\\mathcal{L}(\\boldsymbol{\\omega},\\boldsymbol{\\alpha}|\\boldsymbol{\\{}{\\bf d},\\boldsymbol{\\theta}\\boldsymbol{\\}}^\\textrm{train},\\boldsymbol{\\{}{\\bf d},\\boldsymbol{\\theta}\\boldsymbol{\\}}^\\textrm{val})p(\\boldsymbol{\\omega},\\boldsymbol{\\alpha})\n", "\\end{align*}$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# Training a network" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "#### What are the maximum likelihood estimates of the weights?\n", "
\n", "\n", "\n", "$$\\boldsymbol{\\omega}^\\textrm{MLE}=\\underset{\\boldsymbol{\\omega}}{\\textrm{argmax}}\\left[\\mathcal{L}(\\boldsymbol{\\{}\\boldsymbol{\\theta}\\boldsymbol{\\}}^\\textrm{train}|\\boldsymbol{\\{}{\\bf d}\\boldsymbol{\\}}^\\textrm{train}, \\boldsymbol{\\omega}, \\boldsymbol{\\alpha}^*)\\right]$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "## We degenerate the posterior\n", "
\n", "$$\\begin{align*}\n", " \\tiny\\mathcal{P}(\\boldsymbol{\\omega},\\boldsymbol{\\alpha}|\\boldsymbol{\\{}{\\bf d},\\boldsymbol{\\theta}\\boldsymbol{\\}}^\\textrm{train})\\propto&\\tiny~\\mathcal{L}(\\boldsymbol{\\omega},\\boldsymbol{\\alpha}|\\boldsymbol{\\{}{\\bf d},\\boldsymbol{\\theta}\\boldsymbol{\\}}^\\textrm{train})p(\\boldsymbol{\\omega},\\boldsymbol{\\alpha})\\\\\n", " \\tiny\\to&\\tiny~\\delta(\\boldsymbol{\\omega}-\\boldsymbol{\\omega}^\\textrm{MLE},\\boldsymbol{\\alpha}-\\boldsymbol{\\alpha}^*)\n", " \\tiny\\\\\n", " \\tiny\\mathcal{P}(\\boldsymbol{\\theta}|{\\bf d}) \\propto&\\tiny~\\int d\\boldsymbol{\\omega}d\\boldsymbol{\\alpha}\\mathcal{L}(\\boldsymbol{\\theta}|{\\bf d},\\boldsymbol{\\omega},\\boldsymbol{\\alpha})\\delta(\\boldsymbol{\\omega}-\\boldsymbol{\\omega}^\\textrm{MLE},\\boldsymbol{\\alpha}-\\boldsymbol{\\alpha}^*)\\\\\n", " \\tiny=&\\tiny~\\delta(\\boldsymbol{\\tau})\n", "\\end{align*}$$
\n", "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### All predictions are (unknowably incorrect) estimates

\n", "
\n", "$$\\mathcal{P}(\\boldsymbol{\\theta}|{\\bf d})=\\delta(\\boldsymbol{\\tau})$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### There is no way to interpret how close $\\boldsymbol{\\tau}$ is to $\\boldsymbol{\\theta}$...\n", "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Local maximum likelihood estimates\n", "
\n", "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Are there better methods?" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "## Variational inference\n", "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "$$\\tiny\\mathcal{P}(\\boldsymbol{\\theta}|{\\bf d})=\\int d\\boldsymbol{\\omega}d\\boldsymbol{\\nu}d\\boldsymbol{\\alpha}~\\mathcal{L}(\\boldsymbol{\\theta}|{\\bf d},\\boldsymbol{\\omega},\\boldsymbol{\\alpha})\\mathcal{Q}(\\boldsymbol{\\omega}|\\boldsymbol{\\nu}, \\boldsymbol{\\alpha}, \\boldsymbol{\\{}{\\bf d}, \\boldsymbol{\\theta}\\boldsymbol{\\}}^\\textrm{train})p(\\boldsymbol{\\nu},\\boldsymbol{\\alpha})$$\n", "
\n", "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "#### Still depends on fixed weights in the complex likelihood surface
and choice of variational distribution\n", "
\n", "\n", "$$\\begin{align*}\n", "\\mathcal{P}(\\boldsymbol{\\theta}|{\\bf d})&=\\int d\\boldsymbol{\\omega}d\\boldsymbol{\\nu}d\\boldsymbol{\\alpha}~\\mathcal{L}(\\boldsymbol{\\theta}|{\\bf d},\\boldsymbol{\\omega},\\boldsymbol{\\alpha})\\mathcal{Q}(\\boldsymbol{\\omega}|\\boldsymbol{\\nu}, \\boldsymbol{\\alpha}, \\boldsymbol{\\{}{\\bf d}, \\boldsymbol{\\theta}\\boldsymbol{\\}}^\\textrm{train})\\\\\n", "&\\phantom{hellohello}\\times\\delta(\\boldsymbol{\\nu}-\\boldsymbol{\\nu}^\\textrm{MLE}, \\boldsymbol{\\alpha}-\\boldsymbol{\\alpha}^*)\\\\\n", "&=\\int d\\boldsymbol{\\omega}~\\mathcal{L}(\\boldsymbol{\\theta}|{\\bf d},\\boldsymbol{\\omega}, \\boldsymbol{\\alpha}^*)\\mathcal{Q}(\\boldsymbol{\\omega}|\\boldsymbol{\\nu}^\\textrm{MLE}, \\boldsymbol{\\alpha}^*, \\boldsymbol{\\{}{\\bf d}, \\boldsymbol{\\theta}\\boldsymbol{\\}}^\\textrm{train}).\n", "\\end{align*}$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "# Bayesian neural networks" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Approximate the posterior distribution of weights and hypermeters\n", "\n", "$$\\begin{align*}\n", "\\small\\mathcal{P}(\\boldsymbol{\\theta}|{\\bf d}) =&\\small~\\int d\\boldsymbol{\\omega}d\\boldsymbol{\\alpha}~\\mathcal{L}(\\boldsymbol{\\theta}|{\\bf d}, \\boldsymbol{\\omega}, \\boldsymbol{\\alpha})\\mathcal{P}(\\boldsymbol{\\omega},\\boldsymbol{\\alpha}|\\boldsymbol{\\{}{\\bf d}, \\boldsymbol{\\theta}\\boldsymbol{\\}}^\\textrm{train})\\\\\n", " \\propto&~\\int d\\boldsymbol{\\omega}d\\boldsymbol{\\alpha}~\\mathcal{L}(\\boldsymbol{\\theta}|{\\bf d}, \\boldsymbol{\\omega}, \\boldsymbol{\\alpha})\\\\\n", " &\\phantom{hello}\\times \\prod_i^{n_\\textrm{train}}\\mathcal{L}(\\boldsymbol{\\theta}^\\textrm{train}_i|{\\bf d}_i^\\textrm{train}, \\boldsymbol{\\omega},\\boldsymbol{\\alpha})p(\\boldsymbol{\\omega},\\boldsymbol{\\alpha}).\\phantom{hello}\n", "\\end{align*}\n", "$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "# Everything is dependent on the training data!\n", "\n", "
\n", "\n", "Classical network : $\\tiny \\mathcal{P}(\\boldsymbol{\\omega},\\boldsymbol{\\alpha}|\\boldsymbol{\\{}{\\bf d},\\boldsymbol{\\theta}\\boldsymbol{\\}}^\\textrm{train}) \\to \\delta(\\boldsymbol{\\omega}-\\boldsymbol{\\omega}^\\textrm{MLE},\\boldsymbol{\\alpha}-\\boldsymbol{\\alpha}^*)$\n", "\n", "Variational inference : $\\tiny \\mathcal{P}(\\boldsymbol{\\omega},\\boldsymbol{\\alpha}|\\boldsymbol{\\{}{\\bf d},\\boldsymbol{\\theta}\\boldsymbol{\\}}^\\textrm{train}) = \\mathcal{Q}(\\boldsymbol{\\omega}|\\boldsymbol{\\nu}^\\textrm{MLE}, \\boldsymbol{\\alpha}^*, \\boldsymbol{\\{}{\\bf d},\\boldsymbol{\\theta}\\boldsymbol{\\}}^\\textrm{train})$\n", "\n", "Bayesian networks : $\\tiny \\mathcal{P}(\\boldsymbol{\\omega},\\boldsymbol{\\alpha}|\\boldsymbol{\\{}{\\bf d},\\boldsymbol{\\theta}\\boldsymbol{\\}}^\\textrm{train}) =\\prod_i^{n_\\textrm{train}}\\mathcal{L}({\\bf t}^\\textrm{train}_i|\\boldsymbol{\\theta}^\\textrm{train}_i,\\boldsymbol{\\omega},\\boldsymbol{\\alpha})p(\\boldsymbol{\\omega},\\boldsymbol{\\alpha})$\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## If the training data, network optimisation, or (pretty much) anything else is not perfect, the inference will be biased" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Parameter inference with summaries" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Consider when the likelihood of some data, $\\mathcal{L}({\\bf d}|\\boldsymbol{\\theta})$ is unknown" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "#### Summarise the data, $\\mathscr{f}:{\\bf d}\\to{\\bf t}$, to have a nicer likelihood for the model $\\mathcal{M}\\otimes\\mathscr{f}:\\boldsymbol{\\theta}\\to{\\bf t}$\n", "

\n", "$$\n", "\\mathcal{P}(\\boldsymbol{\\theta}|{\\bf t},\\mathcal{M}\\otimes\\mathscr{f}) = \\frac{\\mathcal{L}({\\bf t}|\\boldsymbol{\\theta},\\mathcal{M}\\otimes\\mathscr{f})p(\\boldsymbol{\\theta}|\\mathcal{M})}{p({\\bf t}|\\mathcal{M}\\otimes\\mathscr{f})}\n", "$$\n", "\n", "#### The data no longer enters the inference, only the summaries" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### We can use a neural network as this summarising function" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "#### But the likelihood of the summaries is still difficult in this situtation" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# Forward modelling" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# Method 1 :

Likelihood-free inference" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "#### Use the same neural network to summarise observations and simulations\n", "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "#### Generate simulations drawn from prior and summarise\n", "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Make a kernel density estimate (or use ABC) on the simulations\n", "
\n", "\n", "A slice through at the summarised observation is the approximate posterior" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Posterior distribution of galaxy counts and fluxes in fields

\n", "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### We're still biased by the form of the density estimator we use, and by the number of simulations we use for approximating the posterior" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# Method 2 :

Infer the data, physics and neural network" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Need a good control on the likelihood of weights and hyperparameters\n", "\n", "#### - remove degeneracies\n", "#### - force to be convex\n", "#### - etc." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "# Use neural physical engines" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "
\n", " \n", "
\n", "

\n", "
\n", "

\n", "
\n", "
\n", " \n", "
\n", "

\n", "
\n", " \n", "

" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# Conclusions" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "### Doing parameter estimation with neural networks is not as easy as predicting parameter values\n", "\n", "### We can still use results from neural networks as highly informative summaries\n", "\n", "### We can also infer all parameters of the forward model, including network to get unbiased results" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# Questions\n", "\n", "### When do we say we trust a neural network?\n", "\n", "### How approximate is too approximate?\n", "\n", "### Are using neural networks for parameter estimation worse than using other summary statistics?" ] } ], "metadata": { "celltoolbar": "Diaporama", "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.4" }, "rise": { "leap_motion": { "naturalSwipe": true, "pointerColor": "#d80000", "pointerOpacity": 0.5 } } }, "nbformat": 4, "nbformat_minor": 4 }